The role of machine learning in developing non-magnetic resonance imaging based biomarkers for multiple sclerosis: a systematic review

Background Multiple sclerosis (MS) is a neurological condition whose symptoms, severity, and progression over time vary enormously among individuals. Ideally, each person living with MS should be provided with an accurate prognosis at the time of diagnosis, precision in initial and subsequent treatment decisions, and improved timeliness in detecting the need to reassess treatment regimens. To manage these three components, discovering an accurate, objective measure of overall disease severity is essential. Machine learning (ML) algorithms can contribute to finding such a clinically useful biomarker of MS through their ability to search and analyze datasets about potential biomarkers at scale. Our aim was to conduct a systematic review to determine how, and in what way, ML has been applied to the study of MS biomarkers on data from sources other than magnetic resonance imaging. Methods Systematic searches through eight databases were conducted for literature published in 2014–2020 on MS and specified ML algorithms. Results Of the 1, 052 returned papers, 66 met the inclusion criteria. All included papers addressed developing classifiers for MS identification or measuring its progression, typically, using hold-out evaluation on subsets of fewer than 200 participants with MS. These classifiers focused on biomarkers of MS, ranging from those derived from omics and phenotypical data (34.5% clinical, 33.3% biological, 23.0% physiological, and 9.2% drug response). Algorithmic choices were dependent on both the amount of data available for supervised ML (91.5%; 49.2% classification and 42.3% regression) and the requirement to be able to justify the resulting decision-making principles in healthcare settings. Therefore, algorithms based on decision trees and support vector machines were commonly used, and the maximum average performance of 89.9% AUC was found in random forests comparing with other ML algorithms. Conclusions ML is applicable to determining how candidate biomarkers perform in the assessment of disease severity. However, applying ML research to develop decision aids to help clinicians optimize treatment strategies and analyze treatment responses in individual patients calls for creating appropriate data resources and shared experimental protocols. They should target proceeding from segregated classification of signals or natural language to both holistic analyses across data modalities and clinically-meaningful differentiation of disease. Supplementary Information The online version contains supplementary material available at 10.1186/s12911-022-01985-5.

9 Specify the methods used to collect data from reports, including how many reviewers collected data from each report, whether they worked independently, any processes for obtaining or confirming data from study investigators, and if applicable, details of automation tools used in the process.
Data items 10a List and define all outcomes for which data were sought. Specify whether all results that were compatible with each outcome domain in each study were sought (e.g. for all measures, time points, analyses), and if not, the methods used to decide which results to collect.
10b List and define all other variables for which data were sought (e.g. participant and intervention characteristics, funding sources). Describe any assumptions made about any missing or unclear information.
Study risk of bias assessment 11 Specify the methods used to assess risk of bias in the included studies, including details of the tool(s) used, how many reviewers assessed each study and whether they worked independently, and if applicable, details of automation tools used in the process.
Effect measures 12 Specify for each outcome the effect measure(s) (e.g. risk ratio, mean difference) used in the synthesis or presentation of results.

Synthesis methods
13a Describe the processes used to decide which studies were eligible for each synthesis (e.g. tabulating the study intervention characteristics and comparing against the planned groups for each synthesis (item #5)).
13b Describe any methods required to prepare the data for presentation or synthesis, such as handling of missing summary statistics, or data conversions.
13c Describe any methods used to tabulate or visually display results of individual studies and syntheses.
13d Describe any methods used to synthesize results and provide a rationale for the choice(s). If meta-analysis was performed, describe the model(s), method(s) to identify the presence and extent of statistical heterogeneity, and software package(s) used.
13e Describe any methods used to explore possible causes of heterogeneity among study results (e.g. subgroup analysis, meta-regression).
13f Describe any sensitivity analyses conducted to assess robustness of the synthesized results.

Reporting bias assessment
14 Describe any methods used to assess risk of bias due to missing results in a synthesis (arising from reporting biases).
Certainty assessment 15 Describe any methods used to assess certainty (or confidence) in the body of evidence for an outcome.

Abstract
Background Background

Item # Checklist item
Location where item is reported RESULTS Study selection 16a Describe the results of the search and selection process, from the number of records identified in the search to the number of studies included in the review, ideally using a flow diagram.
16b Cite studies that might appear to meet the inclusion criteria, but which were excluded, and explain why they were excluded.

Study characteristics
17 Cite each included study and present its characteristics.

Risk of bias in studies
18 Present assessments of risk of bias for each included study.

Results of individual studies
19 For all outcomes, present, for each study: (a) summary statistics for each group (where appropriate) and (b) an effect estimate and its precision (e.g. confidence/credible interval), ideally using structured tables or plots.

Results of syntheses
20a For each synthesis, briefly summarise the characteristics and risk of bias among contributing studies.
20b Present results of all statistical syntheses conducted. If meta-analysis was done, present for each the summary estimate and its precision (e.g. confidence/credible interval) and measures of statistical heterogeneity. If comparing groups, describe the direction of the effect.
20c Present results of all investigations of possible causes of heterogeneity among study results.
20d Present results of all sensitivity analyses conducted to assess the robustness of the synthesized results.
Reporting biases 21 Present assessments of risk of bias due to missing results (arising from reporting biases) for each synthesis assessed.

Certainty of evidence
22 Present assessments of certainty (or confidence) in the body of evidence for each outcome assessed.

Discussion
23a Provide a general interpretation of the results in the context of other evidence.
23b Discuss any limitations of the evidence included in the review.
23c Discuss any limitations of the review processes used.
23d Discuss implications of the results for practice, policy, and future research.

Registration and protocol
24a Provide registration information for the review, including register name and registration number, or state that the review was not registered.
24b Indicate where the review protocol can be accessed, or state that a protocol was not prepared.
24c Describe and explain any amendments to information provided at registration or in the protocol. Support 25 Describe sources of financial or non-financial support for the review, and the role of the funders or sponsors in the review.

Competing interests
26 Declare any competing interests of review authors.
Availability of data, code and other materials 27 Report which of the following are publicly available and where they can be found: template data collection forms; data extracted from included studies; data used for all analyses; analytic code; any other materials used in the review. For more information, visit: http://www.prisma-statement.org/

Figure 1
Results

Discussion
Discussion Conclusion